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Abstract 

Critical thinking is an important goal in higher education. Students are expected to grow in critical thinking 
during their higher education programme. The present study investigates the development of critical thinking in 
1134 bachelor students in Flanders, Belgium. The study followed a mixed longitudinal design. Students’ critical 
thinking was measured by the Scipio, a test consisting of both constructed response items and forced choice 
items. The analyses were split up for academic and professional bachelor students. Students in both academic 
and professional bachelor programmes are shown to become better critical thinkers during the first two years of 
study, although the specific growth patterns of both programme types are different. Professional bachelor 
students show a large growth in the first year of higher education, but do not improve their critical thinking 
during the next two years. Academic bachelors on the other hand show a moderate growth in the first year and 
continue developing their critical thinking during the next years, leading to a higher overall growth than 
professional bachelor students. 

Keywords: critical thinking, development, professional higher education, academic higher education, bachelor 
programmes 

1. Introduction 

Critical thinking (CT) is an important educational goal (Halpern, 1998; Ku, 2009; Tsui, 2002). All disciplines 
and levels of education aim at preparing individuals to think well and to think for themselves (Pithers & Soden, 
2000). CT is claimed to be essential for active and engaged citizenship (Kuhn, 1999; Moore, 2004). It is argued 
to prepare students for a multitude of challenges that they will face in their personal lives and their careers 
(Halpern, 1998; Tsui, 2002). According to Tsui (2002), CT encourages students to become independent lifelong 
learners. Its importance has been emphasized in several European studies as well as studies in the rest of the 
world (Halpern, 1998; Ku, 2009; Pithers & Soden, 2000; Ten Dam & Volman, 2004; Tsui, 2002; Verburgh, 
Schouteden, & Elen, 2013). 

In the research literature, CT is defined in many different ways (Bailin, Case, Coombs, & Daniels, 1999). 
According to Halpern (1998, 1999), CT is the kind of purposeful, reasoned and goal-directed thinking that one 
needs in order to solve problems, make decisions, formulate inferences and calculate likelihoods. According to 
Facione (1990), CT is reasonable and reflective thinking that is focused on the decision what to believe or what 
to do. He defined it as “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, 
and inference, as well as explanation of the evidential, conceptual, methodological, criteriological or contextual 
considerations upon which that judgment is based” (Facione, 1990, p. 3). 

Although there is no absolute agreement about the definition of CT, there is consensus among researchers that 
CT consists of both skills and dispositions (Halpern, 1999, 2007; Ku & Ho, 2010). According to Facione (1990) 
interpretation, analysis, evaluation, inference, explanation, and self-regulation are the major skills composing CT. 
Halpern (1998) constructed a taxonomy of five categories of CT skills: verbal reasoning skills, argument analysis 
skills, skills in thinking as hypothesis testing, likelihood and uncertainty, and decision-making and 
problem-solving skills. However, possessing CT skills is not enough to be a critical thinker. One also needs to 
have an inclination to apply these skills. The disposition towards CT allows a person to recognize when a certain 
skill is required and to apply this skill (Ku & Ho, 2010). The most important dispositions to CT are, according to 
Halpern (2008): the willingness to engage in and persist at a complex task, the habitual use of plans, the 
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suppression of impulsive activity, flexibility, open-mindedness, the willingness to abandon nonproductive 
strategies in an attempt to self-correct, and an awareness of the social realities that need to be overcome so that 
thoughts can become actions. 

Several studies investigated the development of CT in higher education. Most of them found a small increase in 
CT during higher education (Arum & Roksa, 2011; Astin, 1993; Bers, McGowan, & Rubin, 1996; Giancarlo & 
Facione, 2001; Flagedorn et al., 1999; Klein, Benjamin, Shavelson, & Bolus, 2007; McMillan, 1987; Miller, 
1992; A. R. Saavedra & J. E. Saavedra, 2011; Saucier, 1995). Similarly, during higher education a small 
development in epistemic reasoning, which is closely related to CT (Brabeck, 1983), has been established 
(Baxter-Magolda, 2002; Hallet, Chandler, & Krettenauer, 2002; Kitchener, 2002; King & Kitchener, 2004; Wood 
& Kardash, 2002). Flowever, Bok (2006) drew attention to the fact that the established growth in CT in most 
studies is limited. Similarly, Arum and Roksa (2011) highlighted that a significant proportion of students in their 
study showed no growth at all. Pascarella, Blaich, Martin, and Flanson (2011) replicated Aram and Roksa’s study 
and found closely matching results. Aram and Roksa attributed the overall limited growth in CT to students 
becoming more and more academically adrift. Among students in FIE institutions seems to live a culture that 
encourages them to achieve as much as possible with minimal effort. Furthermore, Aram and Roksa claim that 
universities do not pay enough attention to teaching in general. Similarly, Bok (2006) stresses the gap between 
what researchers consider important and the behavior of people in a teaching position. It is however unclear 
whether these conclusions, that are drawn based on research in the US, are valid in the context of European and 
more specifically Flemish higher education. The present study tries to provide more insight into this issue. 

Although most studies found some, be it a limited, growth in CT, it remains unclear how CT exactly develops 
during higher education and at what moment the largest growth in CT takes place. Lehmann (1963) found in a 
longitudinal study that the largest growth occurs during the first year of higher education. Recent research that 
confirms this result is, however, lacking. This is partly due to the difficult character of longitudinal research in 
higher education. Furthermore, in the literature no differences in CT development across higher education 
programmes are reported. 

Evens, Verburgh, and Elen (2013) found differences in CT development during the first year of higher education 
between Flemish professional and academic bachelor students. Professional bachelor programmes prepare 
students for the labour market. Academic bachelor programmes are followed by master programmes. Because of 
the differences in CT that are found between these programme types in the freshman year, it is interesting for the 
present study to investigate whether professional and academic students experience a different development of 
CT in subsequent years of higher education as well. 

The following research questions are investigated in the present study: 

Research question 1: Do professional and academic students grow in CT between the beginning and the end of 
their bachelor programme? 

Research question 2: Do students grow in CT in each separate year of higher education? In which year of 
professional and academic bachelor programmes does the largest growth in CT take place? 

2. Method 

2.1 Overall Design 

The study took place in the educational context of Flanders, Belgium. The development of CT is an important 
and recurrent aim in Flemish higher education programmes and courses (Verburgh et al., 2013), but it has not yet 
been systematically studied. In Flanders, higher education is organized according to the bachelor-master 
structure. Fligher professional education is organized at university colleges and offers bachelor programmes that 
consist of 180 European Credit Transfer and accumulation System (ECTS) points, which requires three years of 
full-time study. Professional bachelor programmes are scaled at 5B medium level in the OECD’s International 
Standard Classification of Education (ISCED-97). Higher academic education is provided by universities and 
includes bachelor and master programmes. Academic bachelor programmes also consist of 180 ECTS points. 
They are scaled at 5A level in the ISCED-97 (Verburgh et al., 2013). 

The study followed a mixed longitudinal design, which is a combination of longitudinal and cross-sectional 
approaches (Van’t Hof, Roede, & Kowalsky, 1977). It was longitudinal in that participants’ CT was tested both at 
the beginning (pre-test) and at the end (post-test) of the same academic year, which enabled to calculate growth 
scores for students who participated twice. Furthermore, the study was cross-sectional because students from 
three years of study were tested at the same time, which allowed to make comparisons over different years of 
study. The design of the study is visualized in Figure 1. Separate analyses were done for professional and 
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academic bachelor programmes because there are differences in growth between the first years of both 
programme types (Evens et al., 2013). In order to facilitate comparability between the academic and the 
professional programmes, only bachelor students were included in the sample. 

Because of the small number of participants in the post-test of the third bachelor year (/? = 54 for the professional 
bachelors, n = 61 for the academic bachelors), the third bachelor students were left out of the sample for research 
question 1. Hence, to find an answer on research question 1 CT performances in the beginning of the first year 
were compared with CT performances at the end of the second year in a cross-sectional design. In research 
question 2 the growth during the first three years was studied in a more detailed way. The growth in CT in each 
year of study was investigated in a longitudinal design. Afterwards, the growth scores were compared with each 
other in order to find out in which year the largest growth in CT took place. 



Figure 1. Design of the study 


Note. Boxes with a bold outline are compared in research question 1. Boxes with a gray background are 
compared in research question 2. 


2.2 Participants 

The sample consisted of 1134 participants. The participants were recruited from 12 educational programmes in 
six institutions of higher education in Flanders. The institutions were taking part in a larger research project that 
focused on research integration in higher education. For this larger project, 18 educational programmes were 
contacted, of which 12 agreed to participate. The remaining six programmes mostly refused because of practical 
reasons (e.g., a lack of available computer rooms). The programmes themselves could choose whether their first, 
second or third bachelor students participated in the test administrations, and whether all the students of a 
particular year of study had to participate or only a few randomly selected class groups. The sample consisted of 
597 first year bachelor students, 330 second year bachelor students and 207 third year bachelor students. There 
were more female (n = 703) than male (n = 428) participants. 630 participants followed a professional bachelor 
programme and 487 participants were academic bachelor students. 976 participants took part in the pre-test and 
639 participants took part in the post-test. 481 participants took part twice. 

2.3 Instruments 

To measure CT the Scipio was used (Verburgh, Francis, & Elen, 2012). This is a Dutch test based on the 
Cornell Critical Thinking Test (CCTT; Ennis, Millman, & Tomko, 2005) and the Halpern Critical Thinking 
Assessment (HCTA; Halpern, 2007). The SCIPIO consists of 17 situations with both constructed response items 
and forced choice items. Two situations were based on the “meaning and fallacies” and “identification of 
assumption” scales of the CCTT. The other items are taken from the HCTA. The test starts with an example 
situation with three questions and possible answers to make participants familiar with the instrument. At the end 
of the test background information is asked, e.g. date of birth, sex, field of study, institution, year of study and 
language proficiency. An example test item can be found in Figure 2. In the Scipio this item is followed by a 
forced choice question on the same topic. 

The maximum score that can be obtained on the Scipio is 95. The maximum score differs per item, ranging 
between one and ten. The answers on the constructed response items were graded with an elaborated scoring key. 
Interrater reliability is high; ranging between .70 and .98 (Verburgh, Francis, Elen, & Janssen, 2013). The 
reliability of the test is moderate (Cohen, 1988) with, depending on the moment of administration, a Cronbach’s 
a between .61 and .67. This result exceeds reliability scores of other CT tests. After translation of the HCTA and 
the CCTT in Dutch, Francois, Verburgh, and Elen (2010) found Cronbach’s a’s ranging between .56 and .72 (for 
the HCTA) and of .52 (for the CCTT). 
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Item 21, Part 1 of the HCTAES test (Question 41) 

Suppose that you are a first-year student at a dental school. You realize that your new friend, who is also a 
first-year student in dental school, is getting drunk on a regular basis several times a week. You do not see any 
signs of her drinking problem at school, but you are concerned because you will both begin seeing patients at the 
school’s dental clinic within a month. She has not responded to your hints about her drinking problem. As far as 
you know, no one else knows about her excessive drinking. 

(A) State the problem in two ways. 

(B) For each statement of the problem, provide two different possible solutions. 


Figure 2. Example item of the Scipio, taken from the HCTA. Adapted from Saiz and Rivas (2008) 


2.4 Procedure 

There were two test administrations, namely in October-November (pre-test) and in May (post-test) of the same 
academic year. Test administrations took place in the participating institutions for higher education. Students 
were asked to fill in the Scipio during lecture time, in order to make sure that all selected students were induced 
to participate. Participants got one hour to complete the test. Mostly the test was taken digitally. In one institution 
a paper version was administered. 

2.5 Data-Analysis 

In order to find an answer on research question 1 independent samples /-tests were performed, in which entrance 
CT performances in the first year were compared with end performances in the second bachelor year. For 
research question 2 pre-test performances and post-test performances in each year were compared with each 
other by means of paired samples /-tests. Hence, only participants who took part in both the pre-test and the 
post-test were included in the analyses for research question 2. Due to this decision selective attrition was 
avoided. When the paired samples /-tests could reveal a significant growth in more than one year of study, 
one-way ANOVA was used to find out in which year the largest growth in CT took place. In these analyses the 
average growth in CT in the subsequent years of study was compared. 

Participants who scored three or more standard deviations from the mean were classified as outliers and removed 
from the sample (Field, 2005). After removing the outliers, 619 professional bachelor students and 482 academic 
bachelor students remained in the sample. The specific distribution of participants per year after removal of the 
outliers can be found in Table 1 and 2. 


Table 1. Descriptive statistics of the professional bachelors (N= 619) 




Pre-test 


Post-test 


n 

M 

SD 

n 

M 

SD 

First bachelor 

Total 

270 

49.63 

8.01 

131 

51.55 

9.62 


Pre + post 

101 

48.37 

7.66 

101 

51.82 

9.74 

Second bachelor 

Total 

172 

51.14 

7.70 

133 

51.98 

7.80 


Pre + post 

104 

52.17 

7.79 

104 

52.11 

7.65 

Third bachelor 

Total 

122 

51.99 

8.04 

54 

51.65 

9.15 


Pre + post 

41 

53.93 

8.10 

41 

52.95 

8.06 


Note. Total = all participants that took part in the pre-test, the post-test or both tests. Pre + post = those 
participants that took part in both the pre-test and the post-test. 
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Table 2. Descriptive statistics of the academic bachelors (N— 482) 



Pre-test 


Post-test 


n 

M 

SD 

n 

M 

SD 

First bachelor 

Total 

255 

50.49 

7.80 

174 

52.64 

7.85 


Pre + post 

140 

50.81 

7.61 

140 

52.58 

7.95 

Second bachelor 

Total 

111 

53.24 

7.21 

76 

55.08 

8.85 


Pre + post 

65 

54.15 

7.18 

65 

56.45 

8.04 

Third bachelor 

Total 

35 

52.06 

7.58 

61 

53.05 

8.01 


Pre + post 

26 

51.15 

8.05 

26 

56.27 

8.45 


Note. Total = all participants that took part in the pre-test, the post-test or both tests. Pre + post = those 
participants that took part in both the pre-test and the post-test. 


3. Results 

3.1 Research Question 1 

Table 1 shows descriptive statistics for the professional bachelor students. The independent samples t-test could 
reveal a significant growth in CT during the first two professional bachelor years. Differences between the 
beginning of the first year and the end of the second year were significant, t(401) = -2.80, p< .01, r = .14. The 
descriptive statistics of the academic bachelor students can be found in Table 2. The analyses indicated that the 
growth between the beginning of the first year and the end of the second year was significant for the academic 
bachelor students as well, t(329) = -4.36, p< .001, r= .24. 

As shown by Figure 3 and 4, the growth during the first two years was larger in the academic bachelor 
programmes (M = 4.59, SD = 1.05) than in the professional bachelor programmes (M= 2.35, SD = .84). As a 
consequence, academic bachelor students (M = 55.08, SD = 8.85), who already had slightly better first year 
pre-test performances than professional bachelors, increased their advantage over the professional bachelors (M 
= 51.98, SD = 7.80) at the end of the second bachelor year. 

3.2 Research Question 2 

As shown in Table 1 and Figure 3, the lowest mean performances of the professional bachelor students are 
obtained by first bachelor students at the beginning of the year (M = 48.37, SD = 7.66). At the end of the first 
year the mean score is higher (M = 51.82, SD = 9.74). At the beginning of the second year the mean score is 
again higher (M — 52.17, SD = 7.79), but during the second year the mean score remains more or less the same, 
leading to a mean score of 52.11 (SD = 7.65) at the end of the year. At the beginning of the third year the mean 
score is 53.93 (SD = 8.10). At the end of the third year the mean performance is again lower (M= 52.95, SD = 
8.06). Paired samples t-tests were performed to check the significance of differences in mean scores for the 
professional bachelor students. Only in the first bachelor year a significant growth in CT took place, i(100) = 
-4.44, p< .001, r = .41. 

As shown in Table 2 and Figure 4, in the first two bachelor years the mean performances of the academic 
students continued to increase. First bachelor students scored on average 50.81 (SD = 7.61) at the beginning of 
the year and 52.58 (SD = 7.95) at the end of the year. The mean scores of second year bachelor students also 
increased between the beginning (M = 54.15, SD = 7.18) and the end (M = 56.45, SD = 8.04) of the academic 
year. In the third bachelor year the mean score was again lower (M= 51.15, SD = 8.05) at the start, but at the end 
of the year the average score increased until 56.27 (SD = 8.45). Analyses indicated that the growth in CT was 
significant during the first (i(141) = -3.19, p< .01, r = .26), the second (7(64) = -2.48, p< .05, r = .30) and the 
third bachelor year (t(25) = -3.56, p< .01, r = .34). The largest growth took place in the third bachelor year (M = 
5.12, SD = 7.33). In order to find out whether the differences in growth were significant one-way ANOVA was 
used. The analyses indicated that there were no significant differences in growth between the years of study, At3) 
= 1.82,/>> .05, co 2 = .01. 
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Figure 3. Bar chart of the mean scores of the 
professional bachelor students by test 
administration 
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Figure 4. Bar chart of the mean scores of the 
academic bachelor students by test administration 


4. Discussion 

Research question 1 examined in a cross-sectional design whether CT grows during the first two bachelor years. 
Both for the professional and the academic students a significant growth in CT during the first two bachelor 
years was found. Professional and academic higher education seem to accomplish the educational goal of 
promoting students’ CT. This finding creates an image that is different from the more negative results of previous 
research about the development of CT in higher education (Arum & Roksa, 2011; Bok, 2006; Pascarella et al., 
2011). This might indicate differences in the educational systems of Europe (Flanders) and the U.S. 

The present study is innovative in studying the differences in CT development between programme types in 
higher education. Academic bachelor students were found to experience the largest growth in CT. They already 
had higher entrance performances than professional bachelor students, so their advantage increased at the end of 
the second year. Before elaborating on the larger overall growth of the academic bachelor students, it is 
interesting to first have a look at the more detailed growth patterns that were studied in research question 2. 

Although both academic and professional bachelor students showed a growth in CT during the first two years of 
higher education, their specific growth patterns were different. Research question 2 studied the growth in each 
year of study in a longitudinal design. Academic bachelor students showed a growth in CT in each of the three 
bachelor years. They showed the largest growth in the third bachelor year. The presence of growth in CT during 
academic bachelor programmes is in accordance with prior research (Astin, 1993; Bers et al., 1996; Giancarlo & 
Facione, 2001; Flagedorn et al., 1999; Klein et al., 2007; McMillan, 1987; Miller, 1992; A. R. Saavedra & J. E. 
Saavedra, 2011; Saucier, 1995). A possible explanation for the finding that the largest growth took place in the 
third year of academic higher education is the idea that the first years of academic higher education emphasize 
reproduction of learning contents more than CT. Dutch research shows that most first year exams do not require 
students’ use of critical, analytical and concrete processing strategies (Vermunt, 2005). This explanation assumes 
that the educational practice in the third year is different. However, the high growth score in the third year could 
also be biased because of the small number of participants in the third bachelor year. Similarly, selection bias 
could play a role; it might be possible that the students that dropped out between the first and the third year are 
the weakest critical thinkers. However, the growth in the first two years is significant, meaning that academic 
higher education seems to succeed in contributing to the development of students’ CT. 

The results of professional bachelor students were different. These students showed a very large growth in the 
first bachelor year, but did not grow significantly in CT during the second and third year. The finding that 
professional bachelor students show a growth in CT during the first two years, which was shown in research 
question 1, is only due to the large growth that occurs in the first bachelor year. The finding that first year 
students experience the largest growth confirms the results of Lehmann (1963). The absence of growth in the 
second and third year is however not in accordance with previous research (Astin, 1993; Bers et al., 1996; 
Giancarlo & Facione, 2001; Hagedorn et al., 1999; Klein et al., 2007; McMillan, 1987; Miller, 1992; A. R. 
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Saavedra & J. E. Saavedra, 2011; Saucier, 1995). Apossible explanation for this finding is the idea that students 
make a major advancement in the first year of higher education because of their very weak starting position. 
Research shows that CT levels of students leaving secondary education are on average low because of the 
production oriented and teacher centered nature of secondary education (de Jager, 2012; Shaila & Traded, 2010; 
Vermunt, 2005). The smaller growth in the second year of professional higher education could be caused by 
habituation of the students to the system of higher education. The absence of a large advancement in the first 
year of academic higher education could be explained by its more production oriented nature and the use of 
larger student groups (Vermunt, 2005). Another explanation for the absence of growth in the second and the third 
professional bachelor year is the possibility that the strongest students of professional bachelor programmes 
transfer to academic bachelor programmes after the first year because they do not feel sufficiently challenged, 
which has a negative influence on the average CT performance of the group of students that does proceed to the 
second year of professional higher education. However, this explanation is not very plausible because in 
Flanders this transition is rarely made by students. 

When comparing both programme types, it seems clear that, although professional bachelor students make a 
larger advancement in the first bachelor year, academic bachelor students show the largest growth in CT during 
higher education. It is unclear whether the advantage of academic students is caused by instructional variables or 
by background characteristics of the students. On the one hand, the fact that academic students have better 
entrance performances in the first bachelor year than professional students indicates an influence of background 
characteristics. Van Houtven, Peters, and El Morabit (2010) for example showed that first year academic 
students outperform their peers in professional bachelor programmes on reading comprehension and 
summarizing, which are skills that play a role in CT (Halpern, 1998). Furthermore, students’ age might play a 
role. Flemish professional bachelor students are on average one year older than academic bachelor students 
(Department of Education and Training, 2009) and research has shown that older students are better critical 
thinkers (Saarmann, Freitas, Rapps, & Riegel, 1992; Tiessen, 1987; Wangensteen, Johansson, Bjorkstrom, & 
Nordstrom, 2010; Yeh & Chen, 2005). This might explain why gains in CT occur earlier in professional than in 
academic bachelor programmes. On the other hand, the finding that academic bachelor students experienced a 
larger growth in CT during the first two years could also suggest an influence of instruction. Curricular 
differences between the programme types might cause a faster initial growth in CT among professional bachelor 
students but a more consistent growth trajectory among academic students. Another possibility is the idea that 
educators understand the concept of CT in a different way in professional and academic education. Verburgh 
(2013) distinguished four specifications of CT in descriptions of higher education teaching staff; CT towards 
oneself, CT towards information, consciousness of the perspective of others and ability to handle uncertainty. A 
different understanding of CT might be a reason why academic and professional teachers, while both aiming at 
promoting students’ CT, attain different results. 

The present study gave a detailed look at the differences in CT growth between professional and academic 
bachelor students. It would be interesting for further research to search for the reasons of these differences, e.g., 
by taking into account students’ background characteristics, effects of maturation, etc. Furthermore, further 
research could use a fully longitudinal design, in which also the third bachelor students and master students are 
included. Finally, there is a need for in-depth analyses that study instruction in both programme types. 

5. Conclusion 

This study examined the development of CT in professional and academic bachelor programmes. In both 
programme types a significant growth over the first two bachelor years was found, which creates an image that is 
different from the more negative results of previous research. The specific growth in the separate years of study 
was different in the two programme types. Professional bachelor students make a large advancement in the first 
year of higher education, but do not grow in CT during the next two years. Academic bachelors on the other 
hand show a moderate growth in the first year, but they manage to continue improving their CT during the next 
years. Hence, in general academic bachelor students perform better than professional bachelor students. The 
importance of the present study lies in the fact that different patterns of CT development were found in different 
programme types. 
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